This notebook gives a detailed explanation of the code contained in input-Consumption.py.
In this notebook, you will manipulate temperature time series, together with models of the thermal sensitivity of consumption.
In the end you will learn how to:
TODO : add something about sectorial decomposition of consumption
import os
if os.path.basename(os.getcwd())=="BasicFunctionalities":
os.chdir('..') ## to work at project root like in any IDE
InputFolder='Data/input/'
#region importation of modules
import numpy as np
import pandas as pd
import seaborn as sns # you might have to isntall seaborn yourself : conda install seaborn
import csv
import datetime
import copy
import plotly.graph_objects as go
import matplotlib.pyplot as plt
from sklearn import linear_model
from functions.f_consumptionModels import * #Il faut préciser le chemin où vous avez sauvegardé les données csv
from functions.f_graphicalTools import * #Il faut préciser le chemin où vous avez sauvegardé les données csv
#endregion
As an introduction, french speakers can read my post (contribution for translation of this site's posts are welcome) on the subject.
You can observe how consumption depends upon temperature for year 2012. Below a threshold temperature, here around 15 degres, the relation between consumption and temperature is linear. The coefficient of the linear regression is, by definition, what we call the thermal sensitivity.
#region Load and visualize consumption
ConsoTempe_df=pd.read_csv(InputFolder+'ConsumptionTemperature_1996TO2019_FR.csv')
ConsoTempe_df["TIMESTAMP"]=pd.to_datetime(ConsoTempe_df['Date'])
ConsoTempe_df=ConsoTempe_df.drop(columns=["Date"]).set_index(["TIMESTAMP"])
year = 2012
ConsoTempeYear_df=ConsoTempe_df[str(year)]
hour = 19
TemperatureThreshold = 15
ConsoTempeYear_df=ConsoTempe_df[str(year)]
plt.plot(ConsoTempeYear_df['Temperature'],ConsoTempeYear_df['Consumption']/1000, '.', color='black');
plt.show()
#endregion
Formally, the preceding linear regression is given by the following formula:
$$ C_t = C^{NT}_t+\rho_{h(t)}(T_t-T_0)_+$$where $T_0$ is the threshold temperature, $(x)_+$ is $x$ for any non negative $x$ and zero otherwise, $\rho_{h(t)}$ is the thermal sensitivity for hour $h$ (if it depends upon the hour of the day) $C^{NT}_t$ is the part of the consumption that is not sensitive to the temperature. It is obtained as the residuals of the linear regression for temperatures lower than $T_0$ and is the consumption itself otherwise.
In module functions.functions_decompose_thermosensibilite, we have implemented a function to estimate the different elements in the preceding equation, given consumption, temperature and other parameters (such as $T_0$). Check the file functions_decompose_thermosensibilite.py to understand how this works.
#region Thermal sensitivity estimation, consumption decomposition and visualisation
#select dates to do the linear regression
#ConsoTempeYear_df.index.get_level_values("TIMESTAMP").to_series().dt.hour
indexHeatingHour = (ConsoTempeYear_df['Temperature'] <= TemperatureThreshold) &\
(ConsoTempeYear_df.index.to_series().dt.hour == hour)
ConsoHeatingHour= ConsoTempeYear_df[indexHeatingHour]
lr=linear_model.LinearRegression().fit(ConsoHeatingHour[['Temperature']],
ConsoHeatingHour['Consumption'])
lr.coef_[0]
#Generic function Thermal sensitivity estimation
(ConsoTempeYear_decomposed_df,Thermosensibilite)=Decomposeconso(ConsoTempeYear_df,TemperatureThreshold=TemperatureThreshold)
#ConsoTempeYear_decomposed_df=ConsoTempeYear_decomposed_df.rename(columns={'NTS_C':'Conso non thermosensible', "TS_C": 'conso thermosensible'})
fig=MyStackedPlotly(y_df=ConsoTempeYear_decomposed_df[["NTS_C","TS_C"]],
Names=['Conso non thermosensible','conso thermosensible'])
fig=fig.update_layout(title_text="Consommation (MWh)", xaxis_title="Date")
#plotly.offline.plot(fig, filename='file.html') ## offline
fig.show()
#endregion
You can do other interesting things. For example, you can redecompose the electric consumption of the year X to thermosensitive and no-thermosensitive parts from the thermosensitivity of the year X and the temperatures of the year Y. It's very useful to compare the years.
#region Thermal sensitivity model to change meteo
## change meteo year
## example for year 2012
newyear=2012
NewConsoTempeYear_df = ConsoTempe_df[str(newyear)]
(ConsoTempeYear_decomposed_df,Thermosensibilite)=Decomposeconso(NewConsoTempeYear_df,TemperatureThreshold=TemperatureThreshold)
NewConsoTempeYear_decomposed_df=Recompose(ConsoTempeYear_decomposed_df,Thermosensibilite,
Newdata_df=NewConsoTempeYear_df,
TemperatureThreshold=TemperatureThreshold)
### loop over years
fig = go.Figure()
TMP=ConsoTempeYear_decomposed_df.copy()
TMP = TMP.reset_index().drop(columns="TIMESTAMP").assign(TIMESTAMP=range(1, len(TMP) + 1)).set_index(["TIMESTAMP"])
fig = fig.add_trace(
go.Scatter(x=TMP.index,y=ConsoTempeYear_decomposed_df['Consumption'],line=dict(color="#000000"),name="original"))
for newyear in range(2000,2012):
NewConsoTempeYear_df = ConsoTempe_df[str(newyear)]
ConsoSepareeNew_df=Recompose(ConsoTempeYear_decomposed_df,Thermosensibilite,
Newdata_df=NewConsoTempeYear_df,
TemperatureThreshold=TemperatureThreshold)
ConsoSepareeNew_df = ConsoSepareeNew_df.reset_index().drop(columns="TIMESTAMP").assign(
TIMESTAMP=range(1, len(ConsoSepareeNew_df) + 1)).set_index(["TIMESTAMP"])
fig.add_trace(go.Scatter(x=ConsoSepareeNew_df.index,
y=ConsoSepareeNew_df['Consumption'],
line=dict(color="#9CA2A8",width=1),
name=newyear))
#plotly.offline.plot(fig, filename='file.html') ## offline
fig.show()
#endregion
In addition, with the same function Recompose to that you can also redecompose the electric consumption of the year X to thermosensitive and no-thermosensitive parts from a new table of thermosensitivity.
#region Thermal sensitivity model to change thermal sensitivity
## change thermal sensitivity
NewThermosensibilite={}
for key in Thermosensibilite: NewThermosensibilite[key]=1/3 * Thermosensibilite[key]
NewConsoTempeYear_decomposed_df=Recompose(ConsoTempeYear_decomposed_df,NewThermosensibilite,
TemperatureThreshold=TemperatureThreshold)
fig = go.Figure()
fig.add_trace(go.Scatter(x=ConsoTempeYear_decomposed_df.index,
y=ConsoTempeYear_decomposed_df['Consumption'],
line=dict(color="#000000"),name="original"))
fig.add_trace(go.Scatter(x=NewConsoTempeYear_decomposed_df.index,
y=NewConsoTempeYear_decomposed_df['Consumption'],
line=dict(color="#9CA2A8",width=1),
name=newyear))
#plotly.offline.plot(fig, filename='file.html') ## offline
fig.show()
#endregion
Now, you know everything about thermosensitivity and you can use theses functions whenever you want in the TP to cast light on interesting values and draw smart conclusions :)
#region multi zone
Zones="FR_DE_GB_ES"
year=2016 #only possible year
#### reading CSV files
areaConsumption = pd.read_csv(InputFolder+'areaConsumption'+str(year)+'_'+str(Zones)+'.csv',
sep=',',decimal='.',skiprows=0).set_index(["AREAS","TIMESTAMP"])
fig = go.Figure()
pal = sns.color_palette("bright", 4); i=0; #https://chrisalbon.com/python/data_visualization/seaborn_color_palettes/
for region in ["FR","DE","ES"]: ### problem with spain data
#tabl=areaConsumption[(region,slice(None))]
fig.add_trace(go.Scatter(x=areaConsumption.index.get_level_values("TIMESTAMP"),
y=areaConsumption.loc[(region,slice(None)),"areaConsumption"],
line=dict(color=pal.as_hex()[i],width=1),
name=region))
i=i+1;
fig.show()
#plotly.offline.plot(fig, filename='file.html')
#endregion
#region Electric Vehicle
VEProfile_df=pd.read_csv(InputFolder+'EVModel.csv', sep=';')
year=2012
NewConsoTempeYear_df = ConsoTempe_df[str(year)]
EV_Consumption_df=Profile2Consumption(Profile_df=VEProfile_df,Temperature_df = NewConsoTempeYear_df[['Temperature']])
fig=MyStackedPlotly(y_df=EV_Consumption_df[["NTS_C","TS_C"]],
Names=['Conso VE non thermosensible','conso VE thermosensible'])
fig=fig.update_layout(title_text="Consommation (MWh)", xaxis_title="Date")
#plotly.offline.plot(fig, filename='file.html') ## offline
fig.show()
#endregion
TODO : if you want to contribute, you can add models in functions.f_consumptionModels.py and explanations here and in input-Consumption.py
#region consumption decomposition
Day="Samedi"
df=pd.read_csv(InputFolder+Day+'.csv', sep=';', encoding='cp437',decimal=",")
Profile_df_Sat=CleanProfile(df,Nature_PROFILE,type_PROFILE,Usages_PROFILE,UsagesGroupe_PROFILE)
Day="Dimanche"
df=pd.read_csv(InputFolder+Day+'.csv', sep=';', encoding='cp437',decimal=",")
Profile_df_Sun=CleanProfile(df,Nature_PROFILE,type_PROFILE,Usages_PROFILE,UsagesGroupe_PROFILE)
Day="Semaine"
df=pd.read_csv(InputFolder+Day+'.csv', sep=';', encoding='cp437',decimal=",")
Profile_df_Week=CleanProfile(df,Nature_PROFILE,type_PROFILE,Usages_PROFILE,UsagesGroupe_PROFILE)
Profile_df_Week["WeekDay"] = "Week"; Profile_df_Sat["WeekDay"] = "Sat"; Profile_df_Sun["WeekDay"] = "Sun"
Profile_df = Profile_df_Week.append(Profile_df_Sat).append(Profile_df_Sun).set_index(['WeekDay'], append=True)
Profile_df_merged=ComplexProfile2Consumption(Profile_df)
Profile_df_merged_spread = Profile_df_merged.groupby(["TIMESTAMP", "type"]).sum().reset_index(). \
drop(columns=["Temperature"]). \
pivot(index="TIMESTAMP", columns='type', values='EnergykWh');
Profile_df_merged_spread
fig = MyStackedPlotly(y_df=Profile_df_merged_spread)
plotly.offline.plot(fig, filename='file.html') ## offline
#endregion